Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add manual #15

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions doc/questions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# ros-get questions

- *1. where do these packages come from??*
- Would be the ros-distro story, but needs a good place in the manual.
- perhaps an example and a eference to the REP.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You create a rosdistro according to REP 143. Example for the tue-robotics software: https://github.com/rayman/rosdistro

I made one modification, the packages field example is added for the source packages field. When this field is present, installation with ros-get is possible.


- *2. which branch?*
- ``ros-get update`` does what wrt branches?
- should it do other things with branches too? (make them, remove them, switch between them, push them, fetch/pull them, etc)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, we should discuss this. My personal preference is the following

  • install
    • if the repo doesn't exist: checkout the repo @ target branch
    • else: just git pull the current branch and warn that you're on the wrong branch
  • update
    • if the repo doesn't exist: checkout the repo @ target branch
    • else: just git pull the current branch and warn that you're on the wrong branch

Does that make sense?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is up to the user to switch branches. And to make sure they're working on the branch they want to work on.
Warnings on an update would be a bit overdone no? If they want to know they can run a ros-get status or something.

- *3. are all branches updated?* Assuming you can have some repos using a different branch, are these updated too, or is that configurable, or?

- *4. is this true?*
- ``ros-get list`` seems to list the top-level packages?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is true, maybe that functionality should be moved to an --installed argument. By default list ALL packages?

- *5. get other information?? how??*
- can it provide other lists (manually installed as required versus installed as dependency, for example)
- can you get packages that nobody needs any more?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The python package catkin_pkg can be used to parse the package.xml files we have on disk and resolve the dependencies. This is indeed work that can be done.


- ``ros-get remove`` considerations:
- *6. really? what if it is needed by some other package??* Can you remove a package that is needed as dependency?
- *7. is a package removed automatically by itself when it is not required any more??* is that desired?
- *8. if so (if automatically removed), can you avoid that?*
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Currently its broken. It should remove top-level packages.
  2. We should have an autoremove command which does that. TODO


# tue-tools differences that may be nice to add

- ``tue-checkout BRANCH`` switch repositories to a branch (would fit in ``ros-get update``? or perhaps ``ros-get branch``?)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is custom for our workflow and will not be in ros-get. We should make a package for this.

- ``tue-get list-installed`` (ros-get list functionality?)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, ros-get list --installed?

- ``tue-get dep [PKG]`` list dependencies of PKG (ros-get list functionality?)
- list upto some depth D?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use rospack depends-on for this, but it will not parse meta-packages. We should look into this in the future.


- ``tue-git-status`` list branch/revision/status of all repos (does not exist yet?)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is custom for our workflow and will not be in ros-get. We should make a package for this.


- ``tue-set-git-remote`` do remote repo management, eg at robocup, add new 'amigo' upstream?
- tue tools seem to modify origin, which seems a bit hackish, perhaps construct a way to switch local master between remote masters?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed for us indeed, I don't know if it belongs in ros-get though

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, not a feature I'd think.


# tue-tools without any clue what it is or does.
- tue-data (no idea wht it is)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some strange svn data thingy, not really needed

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proper bulk-data handling in ROS would be awesome, but not a feature of ros-get indeed. Perhaps a new project if @Rayman gets bored :P

- tue-dashboard (no idea what it is)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a separate package, but it is intertwined with tue-get. Should be refactored a bit to be stand-alone.


# tue-tools differences that we may not want in ros-get
Functionality listed here should either be created by other means, for example as a command, or get discarded

- ``tue-get --release`` build Debian package`` (looks catkin-ish)

- ``tue-make`` / ``tue-make-dev`` / ``tue-make-dev-isolated`` / ``tue-make-system``
- probably provided by catkin already?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make an alias like:

alias tue-make='catkin build --workspace "$(ros-get ws-locate)"'


- ``tue-revert`` and ``tue-revert-undo``.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made these. They revert all packages to a certain point in time. Useful for "Help we need to revert back to yesterday immediately!!"-situations.

Basically high level: ros-get upgrade gives the latest versions. But maybe you sometimes don't want the latest but a specific point in time.

But maybe this is also very tue specific.


- ``tue-create ros-kpg PACKAGE_NAME [DEPENDENCY1 DEPENDENCY2 ..]`` code generator for ROS package boiler-plate
- ``tue-create cpp-class CLASS_NAME PROJECT SUPER_CLASS`` code generator for cpp class.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already catkin_create_pkg

- ``tue-env cd`` woud at least be bash. Likely also exists as ``roscd``?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, roscd is fine, or make an alias. We used to have the trunk alias.

- time-zone commands
- apt-get-proxy
- ``tue-save-map``

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are robotcup specific aliases that should be in their own package.

226 changes: 226 additions & 0 deletions doc/userman.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
# *Ros-get* user manual
The *ros-get* software aims to simplify development of (ROS-based) software by
automating handling of Catkin workspaces and packages therein.


## Catkin workspace
The *ros-get* software builds on Catkin workspaces, this section briefly
explains how such workspaces work.

Global layout of a Catkin workspace is
(see also [Catkin workspaces](wiki.ros.org/catkin/workspaces))

- ``ws`` root directory of the workspace.
- ``src`` Source code of the work space, each package has its own
directory, Catkin constructs a CMake setup directly in ``src`` to build
all the packages in the right order.
- ``build`` Temporary space for ``CMake`` to build the packages from ``src``.
- ``devel`` Built targets are placed here before installation.
- ``system`` Installed targets are placed here.

Both ``devel`` and ``system`` are known as *result* spaces, as they contain
results of the build process.

To use a Catkin workspace, you source the setup script in one of the result spaces.
This sets a number of environment variables, which guide the various Catkin
tools to that workspace.


# *ros-get* tools
The Catkin tools handle building of all packages in the work space, such that
packages are only built if all packages that it needs are already available.

The *ros-get* tools aim to simplify creating a Catkin workspace,
populating it with all the packages that you need from external repositories,
and keeping the packages up-to-date with respect to changes in the
repositories. Also, it has functionality for keeping several workspaces.

Commands related to handling workspaces are
- *ws-create*: Create a new workspace.
- *ws-switch*: Switch to a workspace.
- *ws-save*: Saves the current workspace.
- *ws-list*: List all saved workspaces.
- *ws-locate*: Prints the path to the current workspace.

Commands related to handling packages in a workspace are
- *install*: Install packages.
- *update*: Update all packages in the workspace to the latest version.
- *list*: List all installed packages.
- *remove*: Remove packages.

First, handling workspaces is discussed. Next, handling packages in a single
workspace is considered.


## Handling workspaces
In the ideal case, you have only one Catkin workspace, but as development
progresses you may run into the situation that having more Catkin workspaces
would be useful. For example, when working on a second robot, or making large
changes for example to prepare for a new ROS distribution.

Ros-get has a number of commands to simplify managing several Catkin
workspaces, as explained below.


### *ros-get ws-create*
Creation of a new workspace is one of those things you don't do often enough
to remember all the details. *Ros-get* gives you a command to do this, and also
adds the new workspace in the list of workspaces know by *ros-get*, thus
allowing you to use it with the other workspace commands.
The command is

ros-get ws-create [--name NAME] DIRNAME [EXTEND]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changed on the latest commit :)


The ``DIRNAME`` is the name of an existing (but empty) directory that should
be used as a new Catkin workspace. It creates all the administrative files and
directories in it, using the workspace you have currently selected
with Catkin or *ros-get* as parent. Alternatively, you can specify the parent
workspace that should be used as a second ``EXTEND`` argument. Doing the latter also
has the advantage that you are sure how the new workspace is set up.

Each workspace has a name, which is used with all the workspace commands of
*ros-get*. The name of the new workspace is the same as ``DIRNAME`` (the
directory name of the new workspace) by default, but this can be overridden
with the ``--name`` option.


### *ros-get ws-switch*
With more than one workspace, it is useful to be able to switch between them.
In *ros-get*, that is done with

ros-get ws-switch NAME

where ``NAME`` is the name of the workspace to switch to.
(See the *ws-list* command below if you don't remember the workspace name.)

### *ros-get ws-list*
To get an overview of your available workspaces, use the

ros-get ws-list

command.

### *ros-get ws-locate*
When you are curious which *ros-get* workspace you have currently selected,
you can ask *ros-get* about it by typing

ros-get ws-locate

It prints the path to the current workspace. This command is also useful for
programmatically accessing your current workspace.

### *ros-get* save
To enable *ros-get* for an already existing Catkin workspace, use the
``ws-save`` command, as in

ros-get ws-save [--name NAME] PATH

with ``PATH`` being the root directory of the Catkin workspace. By default,
the name of the new workspace is derived from the directory name. You can
override it by using the ``--name`` option.


## Handling packages
In a single Catkin workspace, the software is organized as a collection of packages
(see http://wiki.ros.org/Packages). Catkin takes care of being able to build
and combine the packages to a functioning piece of software. However, there
are more things to consider with each package.

Packages may come from several sources, and typically depend om other packages
to do their work. Some packages are updated rarely, while others are in more
early development, and get modified a lot. As a result, packages in
a workspace need to be synchronized with their source. Such synchronization
may in turn cause that dependencies between packages change, causing further
changes in the workspace. The latter is what *ros-get* aims to handle using
the commands listed below.


### *ros-get install*
To add new packages to the workspace, use the ``install`` command.

ros-get install PKG1 PKG2 ...

The listed packages get added to the workspace. If they require other packages
that are not yet available, those are added as well.

**1. where do these packages come from??**
(would seem it needs more introduction of how ros-get works?)`


### *ros-get update*
Once you installed a number of packages in your workspace, you can start
working on them, changing them as needed. Likewise, others may update other
packages in your workspace.
Making your own changes available is hard to automate, as it involves a lot of
manual steps. Getting updates from others is however a simple and boring task
that *ros-get* can handle.
The command is

ros-get update

It updates all packages in the workspace from their repositories.

**2. which branch?**
**3. are all branches updated?**


### *ros-get list*
With all the packages in the workspace, it is useful to get an overview of the
top-level **4. is this true?** packages. The list command provides that functionality.
The command is

ros-get list

**5. get other information?? how??**


### *ros-get remove*
Removing packages can be done using the

ros-get remove PKG1 PKG2 ...

which removes packages PKG1 and PKG2 from the workspace.

**6. really? what if it is needed by some other package??**
**7. is a package removed automatically by itself when it is not required any more??**
**8. if so, can you avoid that?**

## Features in tue-env

- ``tue-checkout BRANCH`` switch repositories to the given branch name
- ``tue-get --release`` build Debian package
- ``tue-get list-installed`` list manually installed packages (ie not automagically added dependencies)
- ``tue-get dep [PKG]`` list dependencies of PKG
Perhaps up to some depth D ?
- ``tue-git-status`` list branch/revision/status of all repos

- ``tue-make`` / ``tue-make-dev`` / ``tue-make-dev-isolated`` / ``tue-make-system``
probably provided by catkin already?

- ``tue-revert`` and ``tue-revert-undo``.

- ``tue-set-git-remote`` setup an origin for all packages in the workspace.
Adding a new remote (or deleting one) seems more elementary. Would need
adding a new branch labels for the new remote, although making new branch
labels looks more elementary even without additional remotes.

- Some form of switching main repositories away from github
(``tue-robocup-set-github-origin`` and ``tue-robocup-reset-github-origin``).

- ``tue-status`` dump catkin system dir, tue-env packages, tue repository status ??

- tue-data (no idea wht it is)
- tue-dashboard (no idea what it is)

### Features that we may not want to support

- ``tue-create ros-kpg PACKAGE_NAME [DEPENDENCY1 DEPENDENCY2 ..]`` code generator for ROS package boiler-plate
- ``tue-create cpp-class CLASS_NAME PROJECT SUPER_CLASS`` code generator for cpp class.
Code generators could be a separate set of scripts, probably by extracting
from tue-create (or even keeping tue-create).

- ``tue-env cd`` woud at least be bash. Likely also exists as ``roscd``?
- time-zone commands
- apt-get-proxy
- ``tue-save-map``

25 changes: 13 additions & 12 deletions src/ros_get/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,23 @@ def main():

subparsers = parser.add_subparsers()

subparser = subparsers.add_parser('install', help='install packages')
# Package management commaneds.
subparser = subparsers.add_parser('install', help='Install packages.')
subparser.set_defaults(func='install')
subparser.add_argument('pkgs', nargs='+', metavar='pkg')

subparser = subparsers.add_parser('update', help='update all packages in the workspace to the latest version')
subparser = subparsers.add_parser('update', help='Update all packages in the workspace to the latest version.')
subparser.set_defaults(func='update')

subparser = subparsers.add_parser('list', help='list all installed packages')
subparser = subparsers.add_parser('list', help='List all installed packages.')
subparser.set_defaults(func='list_installed')

subparser = subparsers.add_parser('remove', help='remove packages')
subparser = subparsers.add_parser('remove', help='Remove packages.')
subparser.set_defaults(func='remove')
subparser.add_argument('pkgs', nargs='+', metavar='pkg')

# workspace commands
subparser = subparsers.add_parser('ws-create', help='create a new workspace')
subparser = subparsers.add_parser('ws-create', help='Create a new workspace.')
subparser.set_defaults(func='create')
subparser.add_argument('dir')
subparser.add_argument(
Expand All @@ -33,23 +34,23 @@ def main():
help='Explicitly extend the result-space of another catkin workspace, '
'overriding the value of $CMAKE_PREFIX_PATH.')
subparser.add_argument(
'--name', help='give a name to the workspace, if not given, the name '
'will be inferred by the directory name')
'--name', help='Give a name to the workspace. If not given, the name '
'will be inferred by the directory name.')

subparser = subparsers.add_parser('ws-switch', help='switch to a workspace')
subparser = subparsers.add_parser('ws-switch', help='Switch to a workspace.')
subparser.set_defaults(func='switch')
subparser.add_argument('name')

subparser = subparsers.add_parser('ws-save', help='Saves the current workspace')
subparser = subparsers.add_parser('ws-save', help='Saves the current workspace.')
subparser.set_defaults(func='save')
subparser.add_argument('dir')
subparser.add_argument(
'--name', help='give a name to the workspace, if not given, the name will be inferred by the directory name')
'--name', help='Give a name to the workspace. If not given, the name will be inferred by the directory name.')

subparser = subparsers.add_parser('ws-list', help='list all saved workspaces')
subparser = subparsers.add_parser('ws-list', help='List all saved workspaces.')
subparser.set_defaults(func='list_workspaces')

subparser = subparsers.add_parser('ws-locate', help='prints the path to the current workspace')
subparser = subparsers.add_parser('ws-locate', help='Prints the path to the current workspace.')
subparser.set_defaults(func='locate')

autocomplete(parser)
Expand Down