Project:Big Cheese

Preamble
A Little while ago, the topic came on the mailing list with a link to The LVL1 Hackspace Automation project, named MOTHER. It quickly grew in popularity as a group project, and a few of us ended up discussing it at a Wednesday evening event. We didn't want to take steal the name MOTHER, so we took inspiration from the test print from the vinyl cutter; BeefCheese, and suggested Big Cheese.

I drew out a number of ideas for the system:

Direct Access to Data
Device busses are ultimately connected to a computer via USB ports, serial ports, Ethernet ports, etc. Developers then write software to pull data from the devices, do something to the data and may then send data back out.

The problem with this is the data flow is not controlled, and a greedy script may be polling the network causing massive lag throughout. Serial Port connections can only be maintained to a single data sink so if a developer does not drop their connection to the port then other scripts will fail when trying to interrogate the port.

The result would be an unmaintainable mess. Race conditions from numerous scripts trying to read and write...

Database Centered
A central SQL or OPC database would allow scripts to write data and pull data from a database. Each device would have one script responsible for integrating them to the system; these scripts would acquire timestamped data and dump it into the database. Other scripts could then grab the data and so stuff, then send an actuator command to the device integrator script to control a device on its network.

Automation scripts would just need to access the SQL database (easy enough with most languages), and send a command to the device "driver script"

The driver script could be a compiled dll (or whatever the equivalent is on linux), allowing most languages to connect to it, but interesting issues arise when the dll is being shared between several running instances.

This seems good, but;
 * Each script would need to be either started manually, or automatically scheduled to run by the Op System.
 * The need to send a command/data between scripts is introduced

Big Cheese
The Big Cheese system defines a managed language-independent method for sending data to other scripts, it introduces event driven methods which -if used correctly- remove data polling. It will be slower than direct data access, but if you need to poll data faster than every few 10s of milliseconds, the you are approaching your project in the wrong way.

Abstract
Big Cheese is a proposed software architecture for automating the Hackspace. It has two functions; a message controller for passing system messages between scripts, and a plugin handler for a number of scripts which are either automation scripts or integration scripts.

Integration scripts communicate with specific hardware or other pieces of software. Integration scripts receive data from a source and place them in a data store. They can also receive control messages from an automation script, which will then translate the message to data suitable for sending down the device bus for which it is responsible.

Automation scripts can request data from the database, and act upon the data or they can subscribe to a specific data vale in the database and receive a data change event when the value changes. They can send addressed messages to the message controller, who will then pass their message on to the receiving script.

I have not had experience in designing a plug in architecture for a cross-language application before so a bit of fudging will likely be necessary. I have abstracted out the actual plug-in loading from the main application, as I think this is what will need to be fudged.

I have, however, designed an architecture which used .NET methods to allow external applications to integrate developer written plug ins, and an application which piggy-backed an OPC server to share data.

Big Cheese
The Big Cheese acts as the dynamic owner and message controller for a number of plug in scripts. The application will probably be written in C++, but the protocol for messaging will be based on localhost/loopback TCP/IP and the messages themselves will be in XML. Therefore, plug in scripts may be written in any language that is capable of communicating in TCP.

The Big Cheese can spawn off Cheesy Nibbles which are dynamic handles for each plug in script. Therefore, each plugin is completely independent of the Big Cheese.

With regards the User Interface (UI), I imagine a command line screen with an automatically refreshing status display will be sufficient, although a plugin may well be written giving the application a GUI.

Messaging System
Messages will need to be in a format which can be understood by a number of programming languages, and transmitted in a fairly universal method. The messages are not much of a problem, they could be in flattened strings, such as XML, but the method of transport could prove to be a problem.

 XML-RPC  An oooold protocol, but it does have a client library written in every common programming language. It features XML messages that are sent via HTTP(S) to a server, which sends it on to a waiting client. Pretty much the definition of Big Cheese. It happily supports sending arrays, structs, and anything you want to throw at it in an XML wrapper.

 JSON-RPC  Essentially an updated version of XML-RPC

 .NET methods  Requires all scripts to be compatible with .NET calls

 Memory Map Files  Not suitable for this.

Message Redirects
A destination may change name for whatever reason. The coffee machine may move from the CAN bus to the wireless network. In this instance, it is likely that a number of scripts will all be trying to talk to the old address. A developer can setup a redirect on Big Cheese so any received letters are sent to the registered new address.

In order to prevent a massive load of redirects from bogging down the system, an alert is sent - possibly to an email - to remind someone to change the destination in the script to the new one. Once alerts have stopped being received, the redirect can be left in place or removed.

Safety
It can be expected that there may be some actuators on a network which have a potentially dangerous operation. It is recommended that there is an automation script sending a watchdog signal to the particular actuator, which if not received will send the actuator to a safe state.

High Speed Control
Speaking as a specialist in the field of Measurement and System Control, it is not recommended that a control loop which requires any notable speed be based on the main topology of a given system. A fast controller - such as a PID loop, should be an external peripheral on the network, which exports data readings to the network, and receives setpoint values from the network.

Cheesy Nibbles
As mentioned above, Cheesy Nibbles are handlers for loading dynamic plug in scripts. Each is spawned in its own thread, completely separating it from the main application. Strictly speaking, a Nibble will not communicate directly with the Big Cheese; loaded scripts communicate via a non synchronous messaging system, and the developer is responsible for their script correctly handling messages.

Cheese Monger
The Cheese Monger is the data store and data dispenser for the system. I originally thought about making it a read only data store, which had a controlled method for data mutation. But it would have needed a universal way of allowing plug-ins to read data. So in order for it to fit into the architecture as outlined above, it would need to follow the messaging system.

This document is still only a first draft, and discussions and suggestions are welcome.

Cheesy Parcels
Cheesy Parcels are the particular system messages that a script will need to be able to interpret. They are sent as XML data, which include a destination address, message data,

ReInitialise
Instructs the script to reload its config file

Close
Instructs the script to close any references, release reserved memory and stop

Poll Database
Sends a request to the database to send a particular data value

Register To Data Value
Instead of the script polling the database, it can subscribe to a particular data value(s) and receive a message when that data value changes.