This software is part of a larger suite of microservices designed to remotely manage digital fabrication equipment in a loosely coupled and distributed environment. More specifically, the software in this repo provides a software abstraction layer for 3D printers, laser cutters, vinyl cutters and desktop milling machines and exposes them over the internet as web services that can be consumed through REST APIs.
This software has been developed by Gianluca Cornetta and Javier Mateos within NEWTON project. NEWTON is a large scale Integrated Action, started in March 2016 and scheduled to end in summer 2019, funded by the European Commission under the Horizon 2020 Researh and Innovation Programme with grant agreement n. 688503.
- Preliminary steps
- System architecture
- Machine administration
- Documentation and developer support
- Websites
- Contribution guidelines
- License
Before installing the software you have to make sure that you comply with the hardware and software requirements specified in the next two sections.
This software has been tested on a Raspberry Pi III Model B (amrv7 32-bit architecture) with a 8-GByte SD card. We strongly recommend using this Raspberry Pi model to avoid possible compilation errors of the GPIO module. You need at least 150 MBytes of free disk space to install the software. You also need to connect to your Raspberry Pi a current sensor to monitor the connected machine. You can find sensor specifications here.
The Machine wrapper software requires that you previously install on your system the following software packages:
- Node.js v6.x
- npm v3.10.x
- Mongo DB v3.x
- Cups
We have not tested the software with Mongo DB latest version; however it should work without any problem if you update mongoose to the last version in the `package.json` file with the project dependencies.
To install the Machine Wrapper module go through the following steps:
- download or clone this repo,
- run
npm installin the installation folder, - run
npm run startto start the server.
The server will listen to port 8888 of your Raspberry Pi.
To run the unit test type npm run test in the project root folder. To run end-to-end tests (with selenium) type npm run test-selenium. Please, recall that in order to run tests with selenium you need to install on your test server a browser driver. We recommend using Google Chrome with Chromedriver
The NEWTON Fab Lab software aims at empowering the traditional Fab Lab infrastructure by providing a software abstraction layer that enables inter- and intra-Fab Lab communication and state synchronization through a centralized cloud hub. This, in turn allows:
- exposing remote digital fabrication machines (3D printers, laser cutters,...) as REST web services
- controlling and monitoring remote machine trough web interfaces or programmatically through APIs
- automatic service discovery and fabrication batch routing based on geographic proximity
A NEWTON Fab Lab (see Fig. 1) is implemented as a two-level architecture that can be accessed through a Fab Lab Gateway (that we also call the Pi-Gateway). The Gateway task is forwarding the incoming fabrication requests to one the networked digital fabrication machines. Each networked machine can be accessed through a hardware/software machine wrapper (that we also call the Pi-Wrapper).
The following subsections are dedicated to describe in detail the hardware and software architecture of a NEWTON Fab Lab that has been developed during the development, staging and production phases of the project.
The software architecture is modular and distributed over five layers as depicted in Fig. 2.
The Communication Layer implements the HTTP server and the APIs interface to manage and schedule fabrication batches. The Presentation Layer implements the user interfaces to set up and manage a connected fabrication machine. An MVC (Model View Controller) programming paradigm is used at this stage; namely, a route in the browser triggers a controller function that dynamically generates and renders an HTML view using the data stored in the persistence layer (i.e. data base). The Application Layer implements the business logic. The business logic and the user interface rely on the middleware functions implemented in the Middleware Layer. More specifically, the middleware includes custom and third-party methods to manage security and authentication, machine to machine communications and interfacing, HTML views rendering, system logging, data base connection and access and ADC (Analog to Digital Converter) drivers. Open API 2.0 (formerly known as Swagger) support is integrated in the application middleware, this makes the Machine Wrapper a very developer-friendly software since APIs and data models documentation is embedded into the application, in addition a developer can test the API using the Swagger User Interface that is also embedded in the Machine Wrapper. Swagger Machine Wrapper API specifications (written in YAML) are available in the software root directory. Finally, the Data Layer is used to store session information as well as User and Machine data models. We use a NoSQL model and MongoDB as the data store.
The lack of network connectivity of a digital fabrication machine requires a hardware/software wrapper to overcome this limitation. The wrapper can be implemented using an inexpensive Raspberry Pi embedded computer connected to a machine through its USB port. Digital fabrication machines must operate in a networked real-time environment that must promptly react to any change of machine status. This, in turn, implies that we need somehow to retrieve status information from the machine (i.e., off, busy, idle). Unfortunately, most of the available fabrication machines have proprietary hardware specifications which impede the development of this functionality. However, it is possible to infer the machine status using a very simple workaround that requires the use of a CT (continuous Transformer) sensor and an ADC connected to the Raspberry Pi GPIO that samples the CT sensor at regular time intervals in order to infer the machine status from current drawn by the power supply.
Fig. 3 depicts the fabrication machine set-up and interfacing with both Pi-Wrapper and sensing and signal-conditioning circuits.
The proposed configuration is formed by the following components:
- A Raspberry Pi III embedded computer (i.e., the Pi-Wrapper);
- A split core current transformer sensor (ECS 1030-L72);
- A conditioning circuit to convert the measured current into an input voltage for the Analog to Digital Converter (ADC);
- A Ti ADS 1x15 Analog to Digital Converter.
The ADS 1x15 has an embedded Programmable Gain Amplifier (PGA) and an I2C (Inter Integrated Circuit) bus that can be connected to the GPIO (General Purpose Input/Output) port of the Raspberry Pi. Software support is provided by the Pi-Wrapper middleware layer in which software drivers for the Raspberry Pi I2C pins and the Ti ADS 1x15 are implemented. Supported devices are ADS 1015 and ADS 1115 that are respectively a 12-bit and a 16-bit analog to digital converters; however, the I2C compliance allow seamless operation with converters from other vendors provided a new device driver is added to the Pi-Wrapper drivers’ library.
Fig. 4 depicts ADC interfacing with Raspberry Pi GPIO and sensor with signal conditioning circuit. The resistance and capacity values are reported in Table 1. Resistance R_BURDEN converts the sensor output current into a voltage, whereas resistors R1 and R2 add a DC offset to the output signal. Finally, capacitor C1 is used to stabilize the DC offset added to the signal.
| Component | Value |
|---|---|
| R_BURDEN | 120Ω |
| R1 | 10KΩ |
| R2 | 10KΩ |
| C1 | 47μF |
Each machine in a Fab Lab is wrapped by hardware/software interface that we have called Pi-Wrapper. The Pi-Wrapper is formed by two major software components:
- A specialized machine-to-machine (M2M) communication middleware that allows seamless interaction with the Fab Lab gateway.
- An administration interface that allows setting up and configuring a machine as well as monitoring its status.
This interface runs as a service on the port 8888 of each Raspberry Pi connected to a digital fabrication machine of the Fab Lab. For security reasons the user interface can be accessed only from inside the Fab Lab or the other sites where the Fab Lab network is propagated. By default each Raspberry Pi runs a simple DNS service (Bonjour) configured to resolve the .local domain; thus, each interface can be accessed through a URL like:
http://wrapper_name.local:8888
The wrapper name must be configured in the /etc/hosts file as follows:
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
127.0.1.1 piwrapper
In the configuration above, we have called the wrapper machine piwrapper. Once the Fab Lab network has been configured, the machine web interface can be accessed through the URL assigned to the Pi-Wrapper.
When a user connects to a given machine, he/she is redirected to the interface landing page shown in Fig. 5.
The user can use the form of the landing page to create an administration account for the Pi-Wrapper software. Only one administrator is allowed; thus, if one tries to create another account a warning message will be generated by the System. The machine administrator can go to the login page using the button at the top right of the screen. Fig. 6 depict the interface login page.
Once the user has been logged in he/she is redirected to the dashboard panel (see Fig. 7).
The dashboard is formed by:
- A fully featured VT100 terminal that supports all the BASH commands. The VT100 terminal allows full control of the underlying hardware and the possibility to interact with the machine and the operating system from the web interface.
- A graphic CPU monitor that reports in real-time the load of all the cores of the CPU.
- A panel with system information.
The user can use the sidebar menu on the left side of the dashboard to navigate among the different features of the user interface. The interface allows the user to:
- Configure the machine and the user profile using the Configuration Menu located just below the Fab Lab administrator profile photo.
- Control a machine and submit a fabrication batch using the Control Panel Menu.
- Control the jobs queued in the machine using the Jobs Menu.
- Monitor machine using the Tools Menu.
The configuration menu can be accessed using the dropdown located just below the administrator profile photo. Using this menu, one can either configure the administrator profile or configure low-level features of the machine hardware wrapper.
Fig. 8 shows the configuration panel to edit the administrator profile. One can set the administrator’s contact details, upload a new profile photo or change the login password.
Fig. 9 depicts the configuration panel to edit the machine settings.
This configuration panel offers the Fab Lab administrator the possibility to assign a logical name to a machine, define machine type and vendor in order to configure the correct drivers, and set up the Analog to Digital Converter (ADC) used to sample and convert in digital format the current drawn by the machine to infer its status. Observe that the administrator must define a Threshold current used by the software to decide whether a machine is idle or not. Also observe that the current measured by CT sensor is oversampled by the ADC. The oversampling ratio is defined through two parameters: the Sample time and the Duty cycle (expressed as a percentage of the period of mains supply current). The number of samples is automatically matched to the closest sample rate supported by the ADC device and then averaged. In other words, we have implemented a simple software-defined sliding window digital low-pass filter that rejects high-frequency noise in the AC supply current and improves the measured current accuracy. Also, note that digital fabrication machine connected to the Pi-Wrapper interface is treated as a standard USB printer. Thus the supported formats (.png images for CNC milling machines, vinyl cutters and laser cutters; .gcode for 3D printers) are sent to the machine as raw data over a serial port emulator on USB. The connected device is autodiscovered by the software that displays the device URI in the interface. The user has the possibility to restart the discovery process using the refresh button or to set up a custom baud rate for the interface. The Pi-Wrapper software has a hybrid interface that allows connecting to a digital fabrication machine either through the web interface or through APIs. APIs default parameters can be configured through the machine settings configuration interface as depicted in Fig. 10.
The API parameters dropdown window is machine-dependent and is automatically generated by the software interface once the machine has been specified and set-up by the administrator.
Finally, Fig. 11 depicts the Pi-Wrapper output log relative to the machine discovery process.
First, the MongoDB database instance is deployed, and the connection is carried out, after that the Pi-Wrapper starts the peer discovery process, namely, it looks for an active Fab Lab Gateway. When the gateway is discovered a websocket channel with the peer is established. Finally, the device discovery process starts. In this phase, the Pi-Wrapper looks for a machine connected to it. In the case of Fig. 11, a vinyl cutter is discovered, and a unique connection id code is assigned to the machine.
The Control panel allows configuring a machine and dispatching a fabrication job to it. Fig. 12 depicts a possible configuration of the machine control panel.
The control panel allows configuring the basic machine parameters, set up the head position, selecting the material for fabrication, the machine model type etc. The structure of the control panel depends on the machine type and the interface is dynamically built after the software has been configured to work with a given machine.
Once a fabrication batch has been submitted to a Fab Lab and routed by the Fab Lab Gateway to an available machine, the job is stored in the Pi-Wrapper queue system described. This design choice is fundamentally motivated by two reasons:
- Implement a job scheduling system that allows a fair sharing of the underlying fabrication resources.
- Implement a basic security policy to protect the machine from potentially harmful designs that could damage it.
Once a fabrication batch has been queued, it must be verified and approved by the Fab Lab administrator before being sent to the machine for fabrication. The queued job can be monitored and controlled using the Jobs Menu depicted in Fig. 13.
When a job is submitted, it is assigned a pending status and no priority until it is approved by the Fab Lab administrator.
The Pi-Wrapper user interface also allows the possibility to monitor in real time the machine status as well as system operation. These options are available in the Tools Menu. In this menu, one can select to display on the web interface the system logs (provided the system logger has been configured in debug mode) or to monitor the current drawn by the machine and the device status.
As mentioned before, the status (either off, idle, or busy) is inferred by the current drawn by the machine. Fig. 14 shows the Machine Status Panel. In the panel, a real-time plot of the measured current as well as the machine status are displayed. The machine APIs are also checked performing a GET request at the URL reserved to the device connected to the Pi-Wrapper as shown in Fig. 15.
The Fab Lab software infrastructure has been designed with the developer in mind. For this reason, Swagger has been integrated into the Pi-Gateway and the Pi-Wrapper middleware. This allow the developer to have on-line access to the API documentation and to test the native APIs through the Swagger User interface (Swagger UI). In addition, the API-first approach used to the develop the Fab Lab software allows to easily expand the software, the protocol stack and add new features adding new layers on top of the native APIs without the need of modifying the core software architecture.
Machine wrapper APIs expose methods to add, remove and modify jobs in a machine. These methods can be only accessed from the Fab Lab Gateway (i.e. the Pi-Gateway).
Table 2 displays the resource URI and the implemented HTTP verbs for the Machine Wrapper (i.e. the Pi-Wrapper) APIs.
| Resource | GET | POST | PUT | DELETE |
|---|---|---|---|---|
| /api/login | Error 400 (Bad Request) |
Returns a JWT if login is correct; otherwise displays an error (401 Unauthorized) |
Error 400 (Bad Request) |
Error 400 (Bad Request) |
| /api/jobs | Returns an array with all the jobs |
Submit a job for fabrication |
Error 400 (Bad Request) |
Error 400 (Bad Request) |
| /api/jobs/1234 | Show the status of the job with id=1234 |
Error 400 (Bad Request) |
Updates the status of the job with id=1234 |
Deletes a job if it exists; otherwise displays an error (404 Not found) |
| /api/jobs?user=123&machine=laser%20cutter &process=cut&material=wood |
Error 400 (Bad Request) |
Submit a job to the Machine |
Error 400 (Bad Request) |
Error 400 (Bad Request) |
API versioning is not mandatory for machine wrapper APIs.
Machine wrapper APIs exclusively support JSON format.
The API error codes will match HTTP codes. The following cases are managed:
- Everything worked (success): 200–OK.
- The application did something wrong (client error): 400–Bad Request.
- The API did something wrong (server error): 500–Internal Server Error.
In the case of client and server error, the server will return in the response a JSON object with error details and hints to correct it. The message has the following format:
{code: the error code, message: the error message, details: the error details}
Table 3 reports error codes and details.
| Error Code | Error Details |
|---|---|
| 20 | Machine not found |
| 21 | Bad request |
| 22 | Unsupported file format |
| 23 | mkdir -p error |
| 24 | FIFO error |
| 25 | Missing attachment |
| 26 | Machine update error |
The Machine Wrapper API documentation can be accessed from the Fab Lab network at the following URL:
http://wrapper_name.local:8888/docs
This URL leads to the Pi-Wrapper API documentation landing page depicted in Fig. 16. The lock indicates that these API are secured. API access is guaranteed with a JWT token that is issued by the system to Fab Lab authorized users. Please note that Swagger UI HTML code is linked to external stylesheets and javascript code; thus you must ensure your network has external connectivity in order to use this feature.
In order to use the Swagger User Interface and test the API, a user must get an authorization token first. This can be accomplished with the /login API that accepts a form with username and password (see Fig. 17).
If the authentication is successful, the API returns a response with a 200 status code and the JWT authorization token as depicted in Fig.18.
POST /api/login
Body:
{
"name": "the user password",
"password": "the user password"
}
Response:
200 OK
{
"message": "ok",
"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjU5ZDIxYzUzODk5NTYxMzBlM2JmZjhkYyIsImV4cERhdGUiOjE1MTY3MDI2NjEyNjgsImlhdCI6MTUxNjYxNjI2MX0.b1QKle3wh2LZsbdb8CsMJQLR0a5sopBBUDzlvX0hfVw"
}
POST
/api/jobs?user=1234&machine=laser%20cutter&process=cut&material =wood
Response:
200 OK
{
"jobId": "3a88b824-7268-4468-947a-054f39c86169"
}
Recall, that with this method a design file in PNG format is uploaded on the server. Our API specifications correspond to the HTTP request depicted below.
POST /api/jobs
Host: piwrapper.local/public/uploads
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryqzByvokjOTfF9UwD
Content-Length: 204
------WebKitFormBoundaryqzByvokjOTfF9UwD
Content-Disposition: form-data; name="design"; filename="design.png"
Content-Type: image/png
File contents go here.
------WebKitFormBoundaryqzByvokjOTfF9UwD—
GET /api/jobs
Response:
200 OK
{
"jobs": [
{
"id": "b5136915-cc09-47b4-b97f-8f2e5026af45",
"status": "cancelled",
"queue": "global"
},
{
"id": "5c28a9b3-2eb6-4058-be11-a0682d94090d",
"status": "cancelled",
"queue": "global"
}
]
}
GET /api/jobs/b5136915-cc09-47b4-b97f-8f2e5026af45
Response:
200 OK
{
"job": {
"material": "vinyl",
"switchSort": "on",
"origin": "bottom left",
"diameter": 0.25,
"offsets": 1,
"overlap": 50,
"error": 1.5,
"threshold": 0.5,
"merge": 1.1,
"order": -1,
"sequence": -1,
"power": 45,
"speed": 2,
"xCoord": 50,
"yCoord": 50,
"userId": "the user id",
"jobId": "b5136915-cc09-47b4-b97f-8f2e5026af45",
"status": "cancelled",
"jobPath": "the path to the design file",
"caller": "api",
"createdOn": 1516112089.866
}
}
DELETE /api/jobs/b5136915-cc09-47b4-b97f-8f2e5026af45
Response:
200 OK
{
"message": "OK",
"details": "Job deleted successfully"
}
Please see CONTRIBUTING and CONDUCT for details.
This software is licensed under MIT license unless otherwise specified in the third-party modules included in this package.