Split aspen.py into several packages

[Changaco]

Background: AspenWeb/pando.py#526, AspenWeb/pando.py#527, and #1.

Let's recap. As I see it the plan is:

• aspen—standalone web framework
• Flask-Aspen—Flask integration
• django-aspen—Django integration
• aspen-core—common code for the three above, depends on the two below
• aspen-(dispatcher|fs-routing)—filesystem dispatcher/router
• simplates

[chadwhitacre]

@Changaco We have aspen.{go,js,rb,py} for different language versions. Do you think we should kill off those versions since they haven't gone anywhere? Or should we keep them around in case Aspen starts to get traction and they become interesting to someone again?

If we keep the other language versions around then I think we should stick with the aspen.py naming convention, which I guess would mean aspen.py-core for that one (parallel to aspen.py-plugins), and probably simplates.py.

[chadwhitacre]

I propose dispatcher.py for that piece. Part of the point of factoring it out is to decouple it from Aspen, if anyone wants to use it elsewhere (right?).

[chadwhitacre]

Yielding:

• aspen.py—standalone web framework
• Flask-Aspen—Flask integration
• django-aspen—Django integration
• aspen.py-core—common code for the three above, depends on the two below
• dispatcher.py—filesystem dispatcher/router
• simplates.py

[Changaco]

I'm okay with keeping the non-python repos.

aspen-core.py looks better to me than aspen.py-core. aspen.py-plugins is okay because plugins come on top, but the core is the foundation, not the roof.

dispatcher.py seems too generic to me, I think we need fs and maybe url in that name.

[chadwhitacre]

aspen-core.py
url-fs-dispatcher.py

👍

aspen.py-plugins

I think this repo might go away entirely for the time being. On AspenWeb/aspen.py#527 the intention is to inline the remaining plugins into aspen-core.py, to be split out again downstream of https://github.com/AspenWeb/aspen.py/issues/341. At that point I think we should consider having a separate repo for each plugin rather than a single repo for all plugins. If this "works" we'll see community interest in producing plugins. Multiple repos makes that easier to manage.

[chadwhitacre]

• aspen.py—standalone web framework
• Flask-Aspen—Flask integration
• django-aspen—Django integration
• aspen-core.py—common code for the three above, depends on the two below
• url-fs-dispatcher.py—filesystem dispatcher/router
• simplates.py

[chadwhitacre]

On AspenWeb/aspen.py#527 the intention is to inline the remaining plugins into aspen-core.py

See https://github.com/AspenWeb/aspen.py/pull/527#issuecomment-170975095.

[Changaco]

Okay. Let's talk about Git history. The Flask and Django repos don't need Aspen's history. On the other hand it's logical for aspen.py to keep its Git history, but should aspen-core.py also be based on it? And what about the dispatcher and simplates repos?

If we're going to have several repos containing Aspen's significant history, should we rewrite it first, removing archives from it to make it lighter? If we want to keep the vendor/ directory, I suggest using Git-LFS.

[chadwhitacre]

How about:

1. Land AspenWeb/aspen.py#527.
2. Rename aspen.py to aspen-core.py.
3. Create a new blank repo for aspen.py.
4. Copy/paste http code into aspen.py.
5. Start new repos for everything else we don't have yet.

I'd be fine with Git-LFS.

[Changaco]

It seems weird to me to not have Aspen's history in the new aspen.py.

I agree that the first step is to deal with AspenWeb/aspen.py#527, it needs to be rebased on master, and I need to take a real look at what's in it. I think we should take the Django/Flask parts out of it and put those in their own repos.

My previous comment was misleadingly worded: I think we should lighten the history in any case. We could do something like this: d=$(mktemp -d); git filter-branch -f -d $d --tree-filter 'for f in $(find -type f -name *.tar.gz); do md5=$(md5sum <$f | cut -d" " -f1); size=$(wc -c <$f); list=$(tar -taf $f); echo -e "# The history of this repository has been rewritten to remove tarballs\n# Below is the md5sum, size, and content list of the tarball that was in the original commit\n$md5\n$size\n$list" >$f; done'; rm -rf $d.

[chadwhitacre]

I think we should take the Django/Flask parts out of it and put those in their own repos.

Before or after we land it? @pjz

It seems weird to me to not have Aspen's history in the new aspen.py.

We're essentially forking the current aspen.py repo into two. We can either keep history in one, the other, or both. I don't really care which has the history as long as the history is somewhere.

I think we should lighten the history in any case. We could do something like this:

Works for me. :)

[chadwhitacre]

Before or after we land it? @pjz

... and I were planning to land 527 and then start factoring out additional bits.

[Changaco]

Before or after we land it?

Before. Is there any reason not to do so that I'm missing? It should make the PR easier to review and merge, right?

[chadwhitacre]

Weeeellllll, what we had said is that we didn't want to leave master in an unusable state. Without Flask or Django integration, 527 is unusable.

[Changaco]

Yes, but 527 is going into aspen-core.py, aspen.py should start from the current master (right?) so it'll still be functional.

[chadwhitacre]

It's easy enough to start a second repo with the history from the first (one direction or another). What we can't fork are the issues and pull requests attached to the current aspen.py repo.

@Changaco After the -core fork-out, where should the existing issues/PRs land? On aspen.py or on aspen-core.py?

[Changaco]

I thought about that too but forgot to mention it. I think we should create aspen-core.py instead of renaming aspen.py like you suggested earlier. Either way we'll need to go through open issues and close/move/update when appropriate.

[chadwhitacre]

@Changaco The issue seems to be that the PR that turns aspen.py into aspen-core.py is on the repo that we otherwise want to keep as aspen.py. Right?

[Changaco]

Right, but we don't have to actually merge it in aspen.py, we can easily recreate it in the new repo if we want, it's a single PR.

[chadwhitacre]

Okay, makes sense.

👍

[Changaco]

We should rewrite the history before pushing the new repo though.

[Changaco]

So let's keep working on 527 as if we were going to merge it, but when the time comes we'll do the history rewrite instead, push the rewritten master to aspen.py, and the rewritten just-simplates to aspen-core.py. Right?

[Changaco]

We don't need a vendor/ dir in aspen-core.py, do we?

[chadwhitacre]

@Changaco If weren't not drastically changing our vendoring habits, then I think we do. It's still used in 527, at least.

[Changaco]

Vendoring policy reticketed as #4.

[chadwhitacre]

On #8, @pjz and I made a Core label in aspen.py for all the open issues that should be moved to aspen-core.py once we have that.

[Changaco]

The reason this hasn't moved forward is #4 (and also because I've been busy on Liberapay). Can I get an answer over there on why exactly we vendor dependencies? It's hard to figure out the right solution when the problem is unclear.

[Changaco]

I've rewritten the history to erase the vendor/ directory, force-pushed master in aspen.py, and pushed the rewritten just-simplates branch to aspen-core.py. The aspen-core.py repo is 3 times smaller than the old aspen.py (5.53MiB versus 16.81MiB).

Next step: push the django and flask shims.

[chadwhitacre]

!m @Changaco

[Changaco]

FTR, the HEAD of the old master was AspenWeb/aspen.py@5407264.

[Changaco]

!m @whit537 too, the just-simplates branch was your work. ;-)

[chadwhitacre]

:-)

[Changaco]

I've split the work on shims from 527 in two, and pushed that into the new repos. Next step is to complete them. Shall the license of the shims be MIT like Aspen? (Personally I've been switching my projects to CC0.)

[pjz]

CC isn't meant for software. I vote for GPL but will concede to MIT or Apache if it's what you guys want.

[pjz]

Oh, hunh, looks like CC0 is okay for software since it's essentially a formal way of saying 'public domain'. I'd still vote for GPL, but will go CC0 if you feel strongly that GPL (or LGPL) is the wrong thing.

[chadwhitacre]

I'm -1 on GPL because I don't care for legal activism.

Saying "public domain" is complicated in Germany and Austria (at least). Over in Gratipay we can get away with it for gratipay.com since that's not intended to be used as a dependency. We avoid CC0 for libraries that we want others to use because of the pushback we've gotten from Germans and Austrians. SQLite uses public domain, and they have to create extra hoops as a result. It essentially becomes its own form of a-legal activism.

I prefer MIT to LGPL because it does everything we need it to do, in ~2.6% of the size (NB: the LGPL is an addendum to the GPL).

-rw-r--r--    1 whit537  staff    42073 Mar 11 20:10 GPL+LGPL
-rw-r--r--    1 whit537  staff     1087 Mar 11 20:09 MIT

[pjz]

Sounds like MIT is the consensus.

[Changaco]

I have no specific knowledge of German copyright law, but I suspect it's similar to French one. As far as I know it's impossible to put your work in the public domain in either country, but CC0 already takes care of that: its legal code contains a license fallback.

The reason I ditched the (L)GPL is that people kept asking me to include the full text of the license everywhere (in Git, in PyPI packages, etc), and they're right that the GPL does require that, so I switched to a license that doesn't. Aspen's MIT license also requires including the copyright notice everywhere, and we don't fully respect that: we don't include it in PyPI packages. That being said, I am okay to stick with the MIT license for Aspen projects if you guys prefer it to CC0.

Quite frankly I'm tired of both copyright and copyleft, they just seem to be wasting everyone's time.

[chadwhitacre]

Quite frankly I'm tired of both copyright and copyleft, they just seem to be wasting everyone's time.

Agreed, and we're not the only ones. ;-)

I think our goal is to do the least amount of work that will get the largest percentage of people to not bug us about licensing.

The reason I ditched the (L)GPL is that people kept asking me to include the full text of the license everywhere (in Git, in PyPI packages, etc), and they're right that the GPL does require that, so I switched to a license that doesn't.

Aspen's MIT license also requires including the copyright notice everywhere, and we don't fully respect that: we don't include it in PyPI packages.

Huh, okay. I haven't heard that request before. https://github.com/AspenWeb/aspen.py/pull/550 should solve this for PyPI. What's the "etc"?

[Changaco]

What's the "etc"?

Nothing yet, I just meant there could be others.

[Changaco]

I think our goal is to do the least amount of work that will get the largest percentage of people to not bug us about licensing.

Agreed, and for aspen.py and aspen-core.py the least amount of work is to keep the MIT license.

On the other hand the shims are new packages and we can easily give them a CC0 license if we want, which would save us from having to bundle the license text in these projects.

[chadwhitacre]

I think our best bet is to use MIT for everything:

• Having a single license across all projects under AspenWeb is the least amount of cognitive overhead for us and for our users.
• My recollection is that the German I heard from wasn't satisfied with CC0, and even if he's wrong I'd rather not spend time arguing about it.
• I don't think there is an "etc," because we don't publish our software anywhere other than GitHub and PyPI (do we?).
• Including the license in the distributions we publish to PyPI is trivial.

[Changaco]

Okay, MIT it is then.

[Changaco]

I've just pushed AspenWeb/aspen.py#1. I'm going to sleep now.

[chadwhitacre]

!m @Changaco

[pjz]

So, reading this diff, I really don't like the module name of 'aspen_core'. Why not just 'aspen' ? And make the aspen.py be 'aspen_web' ?

[chadwhitacre]

@Changaco at AspenWeb/aspen.py#1 (comment):

aspen is not "a filesystem router for Python web frameworks", url-fs-dispatcher.py is supposed to be that. The current aspen-core is simplates + fs-dispatch, and eventually it's supposed to be mostly glue code for simplates.py and url-fs-dispatcher.py.

[chadwhitacre]

We decided over on AspenWeb/aspen.py#1 to actually go with Aspen for the core library, and Pando for the standalone framework. Todo (copied from AspenWeb/aspen.py#1 (comment)):

• rename aspen.py to pando
• rename aspen-core.py to aspen.py
• make a PR to change the name in pando: https://github.com/AspenWeb/aspen.py/pull/552
• retitle https://github.com/AspenWeb/aspen.py/issues/548 as "port to aspen"

[chadwhitacre]

aspen is not "a filesystem router for Python web frameworks", url-fs-dispatcher.py is supposed to be that.

That's looking at it from the inside out. From the outside in, "Aspen" is the brand attached to the idea of filesystem web routing in Python. How we carve that up under the hood is another matter.

[Changaco]

A filesystem router could dispatch to python files, or to templates, or to a mixed format other than .spt. Aspen is fs-dispatch + simplates.