Skip to content

RoadMap

Jump to bottom
thebluedirt edited this page Jul 8, 2022 · 20 revisions

CadQuery Roadmap ::DRAFT

This roadmap describes where CQ is heading. Currently, posted for community discussion.

Rules of Engagement/Philosophy

  1. User experience is first

    • Most users are not python experts. Keep things simple when possible
    • Consistency in the codebase is important. We don't want some methods having parameters in snake_case and others in camelCase. Nor do we want parameters with the same name take a letter in some methods, words in another, and enums in a third. They should all be consistent.

  2. Contributing should be easy

    • Volunteers don't have time to understand thousands of lines of code. Make things small and self contained as much as possible.
    • Tests help to document what the code does. Write tests to showcase how methods work and whats supposed to not work. Each pull request should have tests to show what has changed or been added (exception is documentation changed).
    • Volunteers contributed their personal time to write contributions, so we should work hard to accept the work. This means tolerating small syntax issues and fixing them in a later PR, or helping a contributor get the code to the right state. Blocking a contribution may effectively stop contributions from that person forever!
    • Suggested rule for accepting MR/PR: "are things better with or without this MR?" Thanks Antti !

  3. Discussion Etiquette

    • Try to do what's best for CQ, and assume others do too
    • Be open minded
    • Be nice

  4. Keep moving forward

    • We strive for consensus when disagreement occurs. However, core developers will have the final say if consensus can not be reached.
    • We'd rather move forward with an imperfect compromise than to get into analysis paralysis for too long
    • If a participant disappears or refuses to continue the discussion the discussion becomes stale, someone else take action based on the discussion.

Objectives

In CQ 3.0, we seek to accomplish these items. Items are listed in priority order. This roadmap will not producing any breaking changes until the very end (see #1 below), which means that they can all be made on the 2.x stream. The last step of this roadmap is to remove deprecated things, which would be breaking-- so that'd need 3.0. It would be ok to introduce other breaking changes before this roadmap completes, in which case we'd bump to the next major version after that.

  1. Avoid python 2/3 type situation
    • Current fluent api must stay working
    • add new things
  2. Make it easier for novices to use CQ
    • Improved documentation. examples include:
      • more examples, which provide a more gradual learning curve
      • examples that differentiate the fluent api from the underlying operations
      • covering hopefully plugins/extensions that are avalable, so people dont re-invent the wheel
      • clarifying the difference in behavior for the same functions that mean different things in different contexts Sketch.circle, workplane.circle
      • reference models showing worked examples of doing the same object > 1 way
    • repr on lots of things so that print() shows what's going on
    • Direct (non-fluent) api
      • avoid hidden contexts
      • explicit steps and type hints to guide the user
      • handle both wire based and face-boolean operations
      • compatible with constraints
    • Address usability concerns. Examples:
      • Exception handling: should OCC layer exceptions be re-raised, if so , what additional info could be provided?
      • tuple/vector/location
      • magic things
  3. Make it easier to contribute to CQ
    • more modularized code: understand less stuff before you can contribute
    • first-class extensions
      • create seams to allow extensions to contribute code at well defined points: selector, operation/direct api, reference models
      • generate omnibus documentation for cq plus all supported extensions together
      • provide an evolution for extensions to migrate from third party to fully managed by cq, without breaking them as the transition. This is generally how they are done in django
      • [probably using stevedore?])
    • Provide a new, first class, direct ( non-fluent ) api. Discussions have centered on unifying Sketch and Workplane. Has been called called 'Part'
  4. New query/construction tools:
    • support for 'generated by /modified by' queries. This allows finding features that were created by a given operation or tag. called "operation based queries"
    • snap to interesting geometry (mid, center, quadrant, end, tangent, intersection, etc)
  5. Add a modeling context, which can be used to support back-references, tagging, and queries when using the direct api
    • the modeling context contemplated needs to support new queries and backreferences, but can't be construed as a complete replacement for the current fluent context, CQContext. That object should still provide the duties related to fluency
  6. Provide support for future non-OCC CAD Kernels
  7. Provide a roadmap that defines phasing ( but not timing) of implementation.

Coding/Design Opinions

Coding opinions here should be applied to new code in the cadquery project only. IE, none of them are to be considered strong enough to break existing stuff, or to another project/repo.

  1. PEP8/PEP484
    • New apis should be PEP8 (snake case) and PEP484 ( type hints) compliant. The fact that the core isn't is Dave's fault-- old java habits! I'm sorry.
    • Old apis shouldn't be changed just to make them compliant
    • keep pylance and black happy
  2. Try to avoid having things that do the same thing named differently
  3. We should allow code duplication in the interest of reverse compatibility when the risk mitigation is non-trival
  4. Lets try to move to immutable objects and eliminate copies of methods that have a side-affect and non-side affect version: example: moved and move
  5. avoid files > 1000 lines. These tend to work against modularity, which in turn tends to make it hard for new contributors, and discourages modular testing
  6. Lets stay with these things, which seem to be serving well:
    • sphinx (and cq_directive)
    • pytest for tests
    • both conda and pip are good and worth having
  7. when doing versioning, lets follow semver.org
  8. prefer kwargs for functions/methods
  9. Let's try to solve the confusion with Vector, Location, Tuple, etc. Users commonly complain that there doesnt seem to be a standard, so its confusing.

Definitions

Throughout this document, bolded terms are references to this list of definitions.

  • "CAD Kernel" A CAD library, together with some python bindings. Examples: OCP+OCC, parasolid+some bindings, pythonOCC+OCC would be another 'kernel'. CQ is currently built on OCP+OCC CAD Kernel
  • "OCP" the current pywrap project OCP, which provides direct bindings to OCC
  • "OCC" OpenCascade. CQ is cur
  • "Extension" Code in a python package that can be installed by a user to enable new functionality that's usable from within CQ
  • "Fluent api" the currently defined api, mainly Workplane and CQContext classes, which provide the current fluent interface that includes a stack (mostly in cq.py)
  • "First class api" A api designed to be used by 'end users' of CQ. Such an api will include examples, documentation, and test coverage. Extensions should be able to contribute functionality. As of CQ 2.x, the only First Class api is the fluent api.
  • "Neutral api" An api that has no direct references to code specific to a particular CAD Kernel. In the current CQ 2.x framework, that means code that does not directly reference the OCP packages.
  • "Cad Kernel adapter" This term refers to the collection of cq code that accesses the CAD Kernel, and provides interfaces for higher level apis. Currently, this is composed of : (assbembly.py, geom.py, shapes.py, sketch_solver.py, solver,py, and most importers and exporters ).

Current State

Here's a color coded map that shows CQ to the function/method level. I used it to help me get my bearings on how the code exists today.

cadquery current state

  • The right side are Neutral apis, and the left side are things that directly reference OCP
  • file boundaries are the blue boxes
  • Colors are used to show the fluent api, import/export, read, and modify operations. Note how small the fluent api bit is! Note how many copies of red things there are!

cq-existing-situation

Desired, Proposed, far-future end state

cq-where-we-are-going

Proposed Road Map

Proposed steps to accomplish proposed objectives. All of the proposed steps would be done on master branch with no breaking changes, on CQ 2.x stream The last step of the roadmap removes deprecated functionality, culminating in CQ 3.0

Roadmap Steps

  1. Talking: Consensus on Ground Rules and Objectives
  2. Talking then coding: Reference Models and Tests
  3. Talking: Design
    • Modeling Context
    • Extensions
    • Operations
  4. Coding: modeling context
  5. Coding: add part api+deps
  6. Coding: operations:
    • add new plus tests
    • delegate shape* to opearations
  7. Coding: add extensions
  8. refactor kernel adapter
  • add snake case aliases for camelCased things we want to keep
  • mark things to delete deprecated
  1. refactor fluent api
  • add snake case aliases for camelCased things we want to keep
  • mark things to delete deprecated
  1. CQ X.0: remove deprecated

Roadmap steps in detail

These sections describe more about the proposed roadmap stages

Reference Models

Cadquery currently includes approximately 33 examples.

Reference models are a new concept that will improve testability, documentation, and facilitate roadmap discussions. The very first set of reference models will include only fluent api source code, but newer apis and extensions may also provide source that produces the same model.

A "Reference Model" is an object that demonstrates how to use CQ code. A reference model meets these criteria:

  1. A short, unique name, for reference
  2. A short description, describing the main features/functions the reference model is illustrating
  3. A list of design intents.
  4. CQ source code that creates the object using all available First class apis.
  5. Includes test cases for each first class api verifying the object was constructed correctly. Checks include:
    • Overall volume and surface area checksums
    • Selector based assertions that make sense based on the objects
  6. A list of first-class api functions/methods used, in a format that sphinx understands. These will be used to create forward references to the apis for more information. Ideally, they can also be used to create automatic back-references from the source to the reference models that use them.
  7. Expanded text/notes describing techniques or special considerations for the object

CQ documentation will include all of the information above, as well as a rendered object, similar to the existing examples.

Design and architecture discussions about CQ should use reference models to facilitate discussion.

CQ builds will require that all tests for all reference models pass for a successful build.

CQ extensions should contribute additional reference models illustrating use of contributed apis and features

The plans for a extension framework are to allow CQ to generate documentation for all 'standard' extensions. In that case, extension that provide shorter/alternate code to produce a reference object will be included, and grouped with the other approaches to produce it. This is a great way for CQ plugins to demonstrate how much simpler particular scenarios can be using the provided extension, vs using core apis.

Since reference models reference the apis used, in theory it would be possible to produce coverage metrics for how many **first class api **methods/functions are featured in at least one reference model.

Extensions

An extension is a user-loadable python package that contributes new functionality for CQ. Extensions will typically be in 3rd party repositories outside of the cadquery github project, but cadquery may contribute plugins from the central repository as well.

We're imagining 'django-style', very rich extensions that can contribute new functionality at many different layers

Extensions will be implemented using the stevedore library.

Extensions should list some version of CQ as a dependency. All extensions will typically reference, and should prefer CQ Neutral apis.

Extensions may also directly reference CAD-kernel specific code, but will mean that the plugin will need to disable itself if the necessary CAD kernel is not available.

Extension Points

When extensions are implemented, they'll be able to contribute these items automatically when installed:

  • New Operations 2d, 3d or other CAD Kernel operations.
  • New Selectors (
  • New Direct api functions/methods
  • New fluent api methods
  • New Reference Models (and associated tests ), or additional source that produces existing reference models
  • Tests (of course)
  • Supporting code for the above ( supplied by the installed package)

Extensions will be namespaced by the extension name, for example cq_warehouse or cq_more

Documentation

Via stevedore, we should be able to automatically produce 'omnibus' documentation, which includes a complete list of all methods available in both core and known, supported extensions.

Some proposed roadmap items are designed to make it easier for extensions to contribute code, including operations, reference models, and the direct api

Modeling Context

The existing fluent api has a context, which allows the fluent api to manage state as features are built. The context currently also stores tags, which are useful to find previously created items in a chain.

We need to create a new modeling context, for several reasons:

  1. A direct api will not have access to the fluent api's context
  2. The current fluent api context does not have support for storing information about which operation or shape generated other shapes. this is a powerful new feature we'd like to introduce
  3. Moving the modeling context into another place will make things more modular, aiding contributions 1.A place to store context will allow a lot of coding patterns to get back references without the user tracking variables for each item they wish to track.
  4. A modeling context will provide a great place to put a unified query api, so that all of the first class apis can re-use some query/topology traversal code that's currently written in the fluent api

This seems like one class, but its design will be very important.

Eventually, the current fluent api can delegate to this class, to preserve reverse compatibility

Operations

The current fluent api does a good job of avoiding direct use of the CAD Kernel layer. Instead, it uses other packages ( shapes, geom, assembly, sketch, selectors, importers, and exporters ) -- collectively, the CQ "Kernel adapter" , for what it needs to do. Simpler put, it is a purely "Neutral api". This is good! Together, this list of packages makes up the CQ

The Kernel adapter layer has several very important benefits:

  1. Hides complexity of the CAD Kernel from the top level apis
  2. Allows re-using code between multiple top level apis
  3. Provides a lower-level api for direct, low level use, if needed
  4. Provides a place to work around kernel bugs
  5. Allows consistent application of CQ exception handing and topology norms when returning values
  6. Provides a testing seam for unit testing
  7. Allows future additional CAD Kernels to be used by existing top level apis.

For the most part, the existing works well, but there are a few problems:

  1. Since most operations are simply methods on existing (stateful) objects, it is hard for 3rd party extensions to add functionality without monkey patching
  2. The existing files are huge, which makes contributing to them daunting
  3. The objects are a blend of read-only(query) and code that mutates objects.
  4. Many methods act by side affect, which confuses users. , especially when the fluent api is layered on top
  5. There is no support for returning data about which features were created or modified, either by tag or by id, for which we'd like to add support. we'd also like to be able to track features
  6. There is inconsistency with how operations that fail are handled.
  7. Since there's no inheritance structure, its very hard to re-use code
  8. These objects shouldn't store state, such as a modelling context, so we need to look elsewhere to store that. tracking

The proposed operations.py would be a package that contributes a list of 2d or 3d modifications. Each operation would be a single class, maybe something like this:

class BaseOperation(object):
    def __init__(self,id, modelling_context):
        self.id = id
        self.context = modelling_context
        self.shape_log = ShapeLog()
        self.success = False
        self.result_message = None
    
    def get_generated(self):
        return stuff generated
    def get_modified(self): 
        return modified shapes    
    def set_result(self,success,message="OK"):
        self.success = success
        self.result_message = message
        
    def perform(self):
        r.set_result(False,"Not Implemented. Please subclass this. typically, this means create some geometry, add to shape_history, and set result")

Benefits are:

  1. This allows an operation to store a modeling context, and to provide more information after the operation is performed than just a return value. The context allows the operation to put stuff needed into the context.
  2. Gives a place to define and store an operation id, which is required to implement operation based selectors
  3. Allows for a separate phase to validate inputs, and would also allows core routines to link in new operations automatically.
  4. Extensions could provide new operations in the form of new classes, no monkey patching required
  5. Allows using base classes to factor code between:
    • multiple CAD kernels
    • patterns and loops
    • error handling
    • context management
  6. Dramatically increases modularity, vs the existing, 4k line shapes.py. This should make it much easier to accept contributions, since they have smaller blast radius.
  7. Better unit test seam, as now there's an expected test for each operation, and its easy to find the test that goes with the operation

Worth noting: most of the ex sting shape methods are static methods, which means it should be very easy to delegate to the operation version without running into state issues.

When the available CAD Kernel is changed, it is expected that some operations may stop working, and other new ones may become available.

If you refer to the big CQ chart, the operation api should eventually do everything colored RED

Part Api

I hope this will be an api users love. Best way to achieve that? ask users!

  1. Though the fluent api is powerful, it also has a few problems:
  2. It is hard for new users to understand what's going on
  3. Backreferences are hard, which become common when code gets more complex
  4. It makes it hard for extensions to contribute code
  5. There is much confusion when things go wrong
  6. Testing is a nightmare

The current "kernel adapter" is not well documented, and generally isn't a first-class api. Some have argued that the current kernel adapter is already a non-fluent api, and that's true. But discussions of using the current kernel adapter as a new top-level api have generated some thoughts within the community:

  1. There are too many things named nearly the same thing.
  2. There is inconsistency and confusion with copies of methods that sometimes have side affects, and sometimes don't (move/moved)
  3. A new first class api would need some code currently found in workplane ( the fluent api), but some have expressed a desire to unify code between sketch and workplane
  4. There is currently confusing and overlapping code in sketch and workplane
  5. There is a general idea that the current methods have become too complex, one example was raised about lots of methods using clean, combine=true, etc in inconsistent ways

For reverse compatibility, we can't simply go 'fix' these. A practical solution is to build a new api that's designed from the ground up to provide a first class, non-fluent api.

The community has LOTS of ideas about what this api could look like. Many ideas have been proposed.

The main constraints/ starting points for this new api are:

  1. It should not be fluent
  2. It should be a neutral api
  3. It should be designed so that extensions can add functionality, namespaced by the extension name, using stevedore
  4. The api is ready when all of the reference models have been produced using it
  5. It should integrate with the modeling context, so that it can support queries, selectors, etc
  6. modeling operations should produce new objects, not change existing ones. More specifically, it should generate new objects from within the geom.py package for operations, vs modifying existing ones
  7. It should be designed primarily with inexperienced end users in mind. This generally means minimal magic,and accepting a little more code to get the job done in the interest of clarity
  8. this is an opportunity to clean up community sore points with the existing kernel adapter layer, such as
    • tuple vs vector vs location,
    • undesired side affects,
    • inconsistent method names
    • PEP8 and PEP484
    • unify translation and rotation
    • replace combine=true with a combine function
    • add an Axis
    • nicer code using context managers
  9. We can create snake_case aliases for all old camelCase methods, marking old ones deprecated

I'm sure i forgot several features that would be cool. I think sky's the limit here, because we wont break anything.

Release Schedules

CQ doesn't have a predictable release schedule. The community is divided about whether a frequent, fixed interval, on-demand intervale, etc is best, but everyone seems to agree we should do something more than nothing.

We should probably use semver.

A couple of ideas we can consider: Should we define LTS releases? Should we define that major versions are the only releases that can have breaking changes? If so, that would mean that while this project is undreway, all releases are 2.x until we're finally ready to get rid of really old stuff Whatever we decide, the right thing is probably time variant. The right release process will depend on how much activity we have, and how many volunteers we have

I'd recommend talking about this after we've finished discussing these other topics, and have a feel for who all will be slinging code.

Clone this wiki locally