-
Notifications
You must be signed in to change notification settings - Fork 2
/
orphans.tex
135 lines (88 loc) · 12 KB
/
orphans.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
% !TEX root = dagstuhl-eas-manifesto.tex
% ==========================================================
\section*{Orphaned Material}
% ----------
\paragraph{I will actively encourage funding agencies to include software experts in their review processes.}
% \on{was: I will petition/encourage funding agencies to include software experts in the review process for projects/grants that propose to build or use a significant amount of software.}
% The idea being to catch too much software re-implementation, ensure sustainability, etc.
\emph{Background.}
Funding agencies give out a lot of money every year for software that reimplements existing tools, or that suffer from unrealistic software development plans. This severely impacts the sustainability of software that already exists, as it is discarded as opposed to being further developed and improved with new science.
UKRSE\footnote{\url{http://www.rse.ac.uk}}, a community of research software engineers in the UK, is starting to talk to the UK Research Councils (EPSRC\footnote{\url{https://www.epsrc.ac.uk}}, BBSRC\footnote{\url{http://www.bbsrc.ac.uk}}, \etc) about this issue. RSEs were involved in the selection process for the EPSRC RSE Fellowships, which is a good start, but should be extended to all software-heavy grant submissions.
\emph{Contradictions or concerns.}
The ``Not invented here'' syndrome: We don't trust that other group's software.
It doesn't do exactly what we intend to do, and even though it?s 95\% close, we need to start over.
\emph{What actions are needed?}
\begin{itemize}
\item I will recommend research software engineers as members of funding review panels.
\item As a research software engineer I will volunteer to serve on funding review panels as grant reviewers and on funding panels where appropriate.
\end{itemize}
Why would those small actions have impact?
Grant submissions would have to properly consider whether they could reuse existing software and justify fully if not.
Just as we wouldn't expect a research software engineer to comment on the science within a grant, the scientists wouldn't be expected to comment on the software engineering proposed.
Money would be saved by not rewriting lots of similar tools.
Career sustainability would improve because grants could fund the original authors of the re-used software to continue to improve it for other groups.
Who else needs to act?
Research Councils and funding bodies will need to review their processes and adjust to accommodate different (separate?) types of review for grant and project submissions.
% ----------
\paragraph{I will recognize software contributions at conferences by proposing dedicated sessions and prizes.}
%\on{was: I will advocate for recognition of the contributions software authors make to science by proposing sessions on software at conferences and meetings}
%\on{was: I will advocate for recognition of the contributions software authors make to science by working to establish one or more awards for these contributions. }
\emph{Background.} Software has often been an invisible output of research, leading to its being undervalued and its contributions unrecognized.
Few scientific disciplines recognize software contributions through prizes or awards, though offer awards, sometimes many awards, for many other aspects of research in that discipline.
Embedding software sessions into conferences and meetings will make software methods and software authors more visible, essentially reminding people within the discipline that such research enablers and output exist.
Establishing one or more awards for software contributions creates parity between other contributions and that of software.
\emph{Contradictions or concerns.} Some conferences do not provide a natural place to offer a session on software.
Funding and managing the process of awarding a prize for software is a concern.
\emph{What actions are needed?}
Look at award structures that are currently available in disciplines and institutions, and reach out to those who administer these other awards with concrete examples of software contributions that should be showcased. Identify funders and funding opportunities to create and sustain these awards. Reach out to senior software authors nearing the end of their careers to suggest endowing an award. Write proposals for funding one or more awards for software contributions.
Why would those small actions have impact?
Gathering information on award structures and administration will inform the other actions needed to propose awards.
Including discussions about software in conferences and presentations on useful packages, providing a way for people to get more information about software in popular discipline conferences and meetings, and a way for software authors to talk about their concerns and methods, learn more about software engineering, and discuss issues with other software authors and with their software's users helps underscore the importance of computational methods to the discipline.
Who else needs to act?
For holding software sessions at conferences: others in the discipline to propose and/or participate in such sessions and program organizing committees to select/allow such sessions.
Funding organizations to provide money for software awards, discipline societies to administer the process, and people to nominate possible recipients.
% ----------
\paragraph{I will distinguish the intellectual contribution of my software from its service contribution.}
Scientific/academic/research software often contains novel scientific and intellectual contributions that go far beyond infrastructure and should not be artificially distinguished from actual science. I will respond to blog posts claiming otherwise and will produce evidence to support this. When I encounter impressive software projects, I will showcase them on any social media that I use.
% ----------
\paragraph{I will invite developers of software that enables research to be co-authors on papers about that research.}
% ----------
\paragraph{I will publish how I organize and run my software projects.}
% WAS: I will encourage publication about the organizations that are creating scientific software, including which tools they use, their methodologies, how they motivate participation, how they choose code review systems, \etc
%\on{unclear. do we mean process? tools?}
\emph{Background.} The environment in which software is developed, particularly in academia, can shape the reliability, utility, and social impact of that software. This environment includes processes and standards such as communication methods (technical and social), code review (mechanisms, standards and platforms), community engagement (social and professional rewards and incentives), and management structures and strategies.
Improving the awareness of methodologies (successful and unsuccessful) as well as encouraging descriptions of those methodologies can improve the overall efficiency of software development in academia. In particular, there is a vast literature related to software development methodologies in industrial settings. While not all of this literature will directly translate to development of academic software, as incentive, timeline and requirements are often different, this literature can help us to understand how to graft, shape, or invent software practices for academic software.
The WSSSPE conference series\footnote{\url{http://wssspe.researchcomputing.org.uk}} has collected such practices from different communities; disseminating this information to audiences that have not yet seen it will likely increase uptake and awareness.
Technical aspects as well as social aspects can guide the development practices of software in positive or negative directions. For instance, this includes the tenor of communication as well as the medium of communication. Platforms that allow and enable easy engagement and modification as well as appropriate credit (\ie pull requests to DVCS as opposed to patch-based submissions to non-distributed VCS) improve overall sense of engagement on the part of software developers both within and external to the core development team.
\emph{Contradictions or concerns.} Are these part of the ``intellectual contributions'' of the software itself? Do the practices need to find a separate venue (such as WSSSPE)?
\emph{What actions are needed?} Blogging? Emphasizing the ``forgotten'' aspects in project contribution guidelines.
Why would those small actions have impact?
In a project homepage (whatever form that may take), describing the processes and practices can encourage engagement. Describing the reasoning behind those processes and practices will help individuals to decide how to conduct their own projects in the future.
% Leading by example. Practices are important but not recognized as practices that can be designed.
% Who else needs to act: (?)
% ----------
\paragraph{I will acknowledge that reading and understanding source code is a legitimate part of the academic discussion.}
\emph{Background.}
An intellectual contribution cannot be appraised if it is not even observed. Reading source code is at least as hard as reading an academic paper is. This is partly because it actually is an intellectual contribution. To motivate reading source code to others, consider that the quality of their research [method] is directly dependent on the quality of the code they use, similar to the literature they cite to support the background theory on which their research is based.
\emph{What actions are needed?}
\begin{itemize}
\item I will teach people how to read and appraise source code of academic software.
\item I will write source code while considering the future audience of source code readers, which are users and maintainers alike (documenting it with statements of intent and design decisions, not re-narrating the code in natural language).
\item I will show my source code to others pro-actively, rather than waiting for them to look it up.
\end{itemize}
``Use the force, read the source''; that just had to come out.
Education in software and engineering, or programming education, needs to pay attention to the act of reading source code. Here's some educational material:
\begin{enumerate}
\item Crista Lopez; ``exercises in programming style''; a software chrestomathy book that can be used to teach the art of code reading
\item Greg Wilson; ``Beautiful code'', code which is actually very much not perfect, yet had impact and a contribution. Authors not apologizing but emphasizing the qualitative aspects and the impact of the code.
\item Ralf Laemmel; \url{http://101companies.org/}. Comparing code is a necessary skill towards knowing how to read the next piece of code.
\end{enumerate}
% ----------
\paragraph{I will consider and document the sustainability of my research software as thoroughly as its function.}
% \on{Formerly: I will understand what sort of project I'm working with/on. I will help others articulate these choices for their own projects.}
%\on{somewhat mysterious. should be more precise}
%\jv{note the distinction between "sustainable" and "maintainable". A piece of software could be very clean and maintainable, but there might not be the processes or manpower available to sustain it.}
\emph{Background.} Academic software varies widely along dimensions relevant for planning development. Some projects are intended to be long-lived and widely usable; others are intended only to create outputs for a single research project or even paper. Software engineering practices that are appropriate (indeed necessary) for the former may be overkill for the latter.
\emph{Contradictions or concerns.} Even software used for a single paper needs to be correct, auditable and reproducible (if only by the author), necessitating a certain level of engineering effort to ensure quality. And software can evolve from one state to the other as its utility is recognized by others, so these decisions may need to be revisited.
\emph{What actions are needed?}
A framework for understanding different kinds of software and appropriate software engineering and project management techniques would be helpful, as would better ways to evaluate the likely utility of a piece of software.