A word processor knows about words, sentences, paragraphs, pages, and documents. It does not, however, understand the factual information contained in what you read and write. Mimix represents the evolution of the word processor in that it provides a means for text processing software to interpret, display, analyze, compare, correct, and transform the factual data inside our documents.
The Mimix whitepaper describes in detail the historical precedents for a different kind of reading and writing system. It also lays out several scenarios in which such a system can radically transform how we interact with written documents. In short, this demands several new capabilities from our software:
Mimix is a new way of looking at research and writing. To implement it requires new tools and a new vocabulary. When this document first mentions a word that has a special meaning in Mimix, it will appear in bold. All of these terms and many others which are used throughout this document are defined in the Glossary at the end of the paper.
The Mimix ecosystem is built around a new programming language called MSL, Mimix Stream Language. This document describes that language in detail. A new programming language was desirable for several reasons:
Recording a series of text documents and edits in the form of a program allows thinking about an editing session in programming terms. We can view words and paragraphs as variables and values. Text references such as a document we're quoting can be viewed as dependencies. This lets us extract any part of the program (the editing session), find its text dependencies, and work with them using any software that understands MSL.
The MSL language is designed to be easier to write than the actual system code which is needed to perform the same functions. You can think of MSL as a shortcut method for describing reading and writing as they take place in linear time.
In most cases, Mimix users will never see MSL, but it's designed to easy to read by human beings regardless. This makes testing, debugging, and extending the system easier.
The MSL language provides built-in protections for data that the underlying system would not. This is used to enforce protections for factually correct information, which we call canon. MSL also uses a built-in set of rules to prevent overwriting important data. Most importantly, it offers a granular rewind capability in which any piece of data can be "rewound" to any of its previous values. This same ability allows extracting any value in the system for use in a different context with all of its dependencies and sources intact. These features are unprecedented among programming languages and are defining aspects of MSL.
The MSL language is communicated between a viewer , where text editing takes place, and an MSL engine which records the expressions in a hybrid database which offers access to both past and present values. This allows the development of numerous compatible engines and viewers, all of which rely on the present MSL specification.
Anyone can write tools that work with MSL and this is encouraged by specifying a language definition that is simple, complete, and coherent. Developers can easily visualize the MSL they will need to input or output in order to achieve their desired results.
Mimix Stream Language is a domain-specific language designed for recording and playing back text editing sessions. A list of MSL expressions is an indelible record of source materials, original writing, editing, and notes in the order in which they were introduced. MSL is the filament in the Mimix light bulb. It is the most important part of the system.
MSL is designed to be good at two kinds of jobs. First, it can be used to record the actions of a Mimix user as he or she reads, writes, edits, and annotates text. It provides good traceability in that the source of values and their changes over time can be readily ascertained. It enables a rewind facility so that previous versions of a stream can be viewed, extracted, or rolled back.
Second, it can be used as a playback language to recreate the same view or a different view of the recording. This allows recreating an entire view or stream from the MSL text at any point, or creating a new type of presentation or transformation of the materials a user has viewed, written, edited, or annotated (or any subset of these) up to a certain point. In other words, MSL can be used to play back any part of the stream with a different view than the original.
MSL is a stream recording language which captures a user's reading, annotations, writing, and editing into a stream of viewsas they take place in the viewer. User behaviors are recorded as a series of MSL expressions. These can be used individually, in parts, or in a sequence to recreate any part of the stream using the same view, a different view, or a different viewer.
MSL expressions are executed by a state machine. The state is stored in a hashtable in the MSL engine. MSL expressions passed to the engine can get or set values in the hashtable. The current value of the hashtable represents the full system state after the last MSL expression executed. The state machine serves as a convenient reference to the values contained in the stream, which are also preserved separately.
MSL is an imperative language in which every expression recalls or updates the system state in a sequential order. The state is defined as the value of the hashtable up to and including the execution of the last MSL expression. Although MSL is not procedural, it does allow the application of selectors and transforms , functions which are applied to an atom, the basic unit of data in the language. Selectors and transforms permit viewers and engines to define functions outside of MSL which are then acted upon by MSL values.
MSL is lexically scoped with atoms having values defined in the atoms themselves, their metadata, in their applied canon, or in one of these belonging to a previous containing view, stream, world, or machine.
MSL is a functional language in which every program contains all its literal values inline and all functions can be resolved through variable substitution without reliance on an independent state. MSL is also functional in the style of the Lambda Calculus in that every MSL expression is permitted only one parameter, its value. MSL supports currying values in that the result of single-argument functions are passed upwards to other functions in order to provide a final value.
The hybrid database in a Mimix system serves only as a convenience for quick resolution of an atom's value. The actual values themselves are always recorded in MSL text and can be resolved by simply examining the MSL text backwards without reference to the hybrid database.
MSL is not Turing complete and essentially consists of only elaborate variable getters and setters. No MSL expressions perform looping, testing, branching, or recursion.
MSL is an interpreted language with the system state stored in a nested series of hashtables backed up by a stream of MSL text. MSL expressions which set an atom's value change the system state by updating values in the hashtables and recording a new entry in the text. MSL expressions which only return a value do not change the system state and simply recall the current value from the hashtables or the text while also recording the expression itself in MSL text.
The simplest way to provide a permanent record of what you read and write is to write down everything you do in chronological order. In fact, this is how MSL works.
If you were planning a trip to Hawaii, you might read two books and make some notes:
(@fodor-book Fodor's 2020 Guide to Honolulu and Oahu) (@lonely-planet Lonely Planet Hawaii) (@hawaii-notes International Marketplace. Pearl Harbor. Shave ice.)
These are MSL expressions. Whenever a valid MSL expression appears in this document, we'll use a code font in blue. When several expressions are given as examples of the same concept, we'll number that list. When MSL concepts are referenced without a full, valid syntax, we'll use a code font in black.
The (@)notation here refers to an atom , the basic unit of storage in MSL. Every atom includes a key which identifies it. It's customary (but not required) to write the key up against the @ sign.
(@lonely-planet Lonely Planet Hawaii)
This atom contains the title for a book. In order to be useful as a reference going forward, we would need to record the entire book. We could simply dump the book's contents into the atom. Or we could give individual paragraphs, people, places, etc. their own atoms that sit inside the book atom.
(@lonely-planet Lonely Planet (@Hawaii)) (@intro The goddess (@Pele) …) (@chapter-1 (@p1 When the ancient (@Hawaii Hawaiians)…) (@p2 …)) (@chapter-2 …))
Text which is read, written, or edited by a Mimix user is recorded sequentially in a series of atoms, and this will be shown through many examples.
Atoms can be grouped into views, streams, worlds, and machines. Atoms can also be defined in canon, giving them a type of data protection. Views, streams, worlds, machines, and canon are atoms that reside in their own namespaces. These basic concepts enable all five of the abilities we need to go beyond word processing.
When a user browses, creates, or edits written materials in Mimix, an MSL viewer records those actions by writing MSL expressions into a stream and passing them incrementally to the engine. As the user works, the engine records the stream as MSL text files.
The viewer creates a user interface from MSL expressions. It may offer a full word processing environment with facilities like text editing, multiple document views, importing, exporting, encryption, stream management, and verification or substitution of factual data. Alternatively, it may offer a more limited set of interfaces. A specialized viewer could be developed which only translated language text, for example. Or a different viewer might consume a specific type of documents, like legal documents, and output them as MSL for an engine.
An MSL engine accepts MSL expressions and gets or sets the corresponding values from the internal hashtable and the existing recorded streams which together comprise the system state. It has no other externally accessible functions. The MSL engine only accepts MSL as input and only provides MSL as output. Any part of the current or previous system state, from the value of a single atom to the full set of all values from the machine, can be requested by an MSL expression and returned as an MSL expression. The "requesting" expression has keys without values and the "response" expression has the exact same form with values provided after each key name.
The hashtable is strictly a convenience. It provides a faster way to find the current value of any atom. The MSL text, however, contains everything known about the system so the hashtable isn't technically required. The engine uses the MSL text to find previous values which might no longer be in the hashtable when those are requested by the viewer.
Decoupling the engine from the viewer allows any number of MSL viewers to be developed with unique capabilities and user interfaces. Several viewers can connect to the same engine at once and that engine can be running on the local machine, on a network, or in the cloud. For details on how MSL applications communicate using MSL, see Nebula: Simple & Flexible Apps with MSL Data.