¶ Nebula Testbed
The Nebula Testbed is a robust environment for test-driven development and debugging of MSL applications. When used with a library of test files (JSON) and MSL text files (MSL), the Testbed provides numerous ways to design and debug MSL applications.
NOTE: LOCAL FILES
The current version of Nebula uses browser-based (rather than Node.js) file handling. This imposes a restriction on file locations:
- JSON test files can only be loaded from
install-location/mimix/json- MSL text files can only be loaded from
install-location/mimix/mslTo load a file, place it in the appropriate directory and navigate there from the load dialog. To roundtrip your changes, save your edited test lists and MSL text into the appropriate directory when prompted by the save dialog.
¶ Connection Controls
The connection controls at the top of the page show the status of Nebula's connection with a streams server. From left to right, they are:
- Version of the Mimix World currently displayed.
- Version of the connected Mimix Stream.
- An MSL wire is connected.
Connect to the Stream's MSL wire.- An admin wire is connected.
Connect to the Stream's admin wire.- Shared History is turned on.
Shared History is off.- Debug Panel is showing.
Debug Panel is hidden.
¶ Testbed Concepts
The Testbed consists of five main parts, not all of which are displayed at once:
- List Controls
- History Rows
- Editing Panel
- Test Strip
- MSL Text
¶ List Controls & History
The Testbed shows MSL and admin messages as a list of tests, with one message sent per test on either the MSL or admin wire. Tests go beyond simply sending messages by tracking the history of each message and allowing comparison with expected results from each wire. Tests can be chained together and run as a series, or breakpoints can be inserted anywhere in the list. Tests can also be edited and the entire list can be saved to or loaded from a JSON file. More information about test list files can be found in the Development section.
¶ Test Editing
To facilitate debugging, extensive test editing controls are provided. Tests can be added, edited, run, or reset while editing.
¶ Regression Test Coverage
To facilitate test-driven development, an integrated test strip can be used like a litmus test to visually indicate the MSL language features which are covered by a particular test. Test numbers can be generated by clicking bits in the test strip and pushed to a new test being written. Test numbers in the history can be used to recall the test to the strip and light the appropriate bits to show its coverage. More information about test strip files can be found in the Development section.
¶ MSL Text Files
Finally, the Testbed includes functions for reading and writing the test list as an MSL text file, a plain text file with one MSL expression per line. Controls are provided to convert MSL text to a test list and vice-versa. More information on MSL text files can be found in the Development section.
¶ List Controls
The list controls row consists of nine parts, left to right:
- Test count. Shows the number of tests on the list.
- Run the entire test list from row #1.
- Reload the test list file.
Shift Choose a JSON test list file to load.- Add an empty test at the end of the list.
Ctrl Add an empty test at the beginning.
Shift Empty the test list.- Save the test list template to disk without results.
- Save the test list with result history to disk.
- Clear the results from every test on the list.
- Open the MSL Text editor.
- Column headers. Provide titles for test messages and results. After running a test, the MSL Received and Admin Received headers light green if all tests in that column passed and red if any failed. When editing a test, this area shows the row number being edited.
¶ History Rows
When testing with the Testbed, the history is a list of tests originally loaded from a file. If any tests on the list have been run, the history includes each test's results. Because the test list can include both MSL expressions and admin messages, Shared History is always on for the Testbed page.
Each history row consists of 10 parts, left to right:
- Row number. Shows the order in which tests will be performed.
- Start the test list from this MSL expression.
Start the test list from this admin message.- Set a breakpoint on this row.
Clear the breakpoint on this row.- Open the editing panel.
- Test number. Used with the test strip in showing regression test coverage. Click to send this row's test number to the strip. Lights green if this row's test number matches the strip's test number.
- Send. Shows the MSL expression or admin message that will be sent when the test is run.
- MSL Expected. Shows the MSL that's expected to be received from the send in order for this test to pass. MSL comments are ignored.
- MSL Received. Shows the actual MSL expression received when the test is run. Lights green if it matches MSL Expected, red if it does not, and cyan if MSL Expected is empty.
- Admin Expected. Shows the admin value that's expected to be received from the send in order for this test to pass.
- Admin Received. Shows the actual admin value received when the test is run. Lights green if it matches Admin Expected, red if it does not, and cyan if Admin Expected is empty.
¶ Editing Panel
While tests can be edited without opening the panel, clicking the icon on any history row provides larger boxes for editing and additional controls for working with rows. The editing panel includes 16 controls and indicators, including 8 row controls, 1 set of potential row warnings, and 7 expanded editing boxes:
¶ Row Controls
- Row number. Shows the order in which this test will be performed.
- Start the test list from this MSL expression.
Shift to switch to Admin type.
Start the test list from this admin message.
Shift to switch to MSL type.- Set a breakpoint on this row.
Clear the breakpoint on this row.- Close the editing panel.
- Copy this row below and edit the copy.
- Remove this row.
- Move this test up one row.
Shift to move to top.
Ctrl to move down.- Add an empty test on this row.
Shift Empty this test.
Ctrl Reload only this test from file.
¶ Row Warnings
- appears if the row is missing notes or if the row is missing a regression test number.
¶ Editing Boxes
- Test number. Compose or edit the regression test number. Click the test number on the strip to copy that number to this box. Lights yellow.
- Notes. Type notes to help remember the purpose of this test, especially to differentiate between instances of the same regression test number.
- Send MSL or Send Admin. Edit the message that will be sent when this test is run. To switch types, Hold Shift and click or on this row.
- MSL Expected. Edit the MSL expression that is expected for this test to pass. Leave blank for "Don't care." Admin messages do not return MSL results.
- MSL Received. Shows the actual MSL expression received after the test is run. This box cannot be edited. Lights green if it matches MSL Expected, red if not. Lights cyan if MSL Expected is empty.
- Admin Expected. Edit the Admin message that is expected for this test to pass. Leave blank for "Don't care." Both MSL and admin messages return admin results.
- Admin Received. Shows the actual admin message received after the test is run. This box cannot be edited. Lights green if it matches Admin Expected, red if not. Lights cyan if Admin Expected is empty.
¶ Test Strip
The integrated test strip is designed to encourage test-driven development by providing a visual indicator of the language features which are covered by each test. Making changes to the strip highlights all the tests on the list which match the strip settings. If one or more tests on the list match the strip, all those test numbers light green. If no tests on the list match, the generated test number lights red.
At the present time, test numbers are managed manually. In other words, it is up to the developer to confirm that the actual features covered in an MSL expression are the same as those indicated by the strip.
¶ Test Numbers
The 8-digit dashed test number is a decimal representation of the individual bits which are turned on (or lit) in the test strip. Together, these represent the language features covered by the test.
Note that regression test numbers do not indicate the actual syntax of any particular MSL expression. Many different expressions can be used to test the same features and all of these will have the same regression test number.
#0839-0656 (@WALT :wife //key meta)
#0839-0656 (@LILLIAN :birthday //key meta)When a test number is shown on the strip, all matching numbers in the list below are lit green.
¶ Test Strip Debugging
When the mxDebug window is open, an additional row of information appears below the strip. Click the binary number in this debug row to reset the strip to its default values as specified in the test-strip.json.
¶ Coverage Concepts
¶ Test Bits
The test strip consists of 24 individual bits, defined in install-location/mimix/json/mimix-test-strip.json. If a bit in the test strip file has notes, hovering over that bit shows the notes. If a bit is turned on, it lights up. The test strip file can also define bits that are required or prohibited by other bits and these are handled automatically as you click around the strip.
¶ Atom & Metadata Bits
Six bits are used to test coverage for atoms, values, regex selectors, bracketed transforms, datatype selectors, and format transforms. A second set of 6 bits covers the same tests for metadata.
¶ Special Case Bits
For each of the 12 primary bits, a second adjacent bit marked x indicates a special case of the preceding MSL feature, described in the bit listing below. Because each special case is also an example of the regular case (ie: a special case regex selector is also a regex selector), the corresponding regular case bit cannot be turned off while a special case bit is on.
¶ Atom Bits
- @ Atom. Always on.
(@WALT)- x Atom special case. Undefined.
- = Value.
(@WALT Walt Disney)- x Value is an embedded atom.
(@WALT (@WED Walter Elias Disney))- // Regex.
(@WALT Walt Disney /Walt/ Walter Elias)- x Regex chaining or flags.
(@WALT Walt Disney /Walt/ Walter /Disney/ Elias Disney)- [] Bracketed transform.
(@WALT [//drive/file.msl])- x Bracketed transform multiple occurrences.
(@WALT [//drive/file.msl] [//drive/file2.msl])- d Datatype selector.
(@WALT (d person))- x Datatype multiple occurrences.
(@WALT (d person) (d inventor))- f Format transform.
(@WALT (f timeline))- x Format chaining.
(@WALT (f published-works) (f timeline))
¶ Metadata Bits
- : Metadata key.
(@WALT :wife)- x Metadata special case. Undefined.
- = Metadata value.
(@WALT :wife Lillian Disney)- x Metadata value is an embedded atom.
(@WALT :wife (@LILLIAN))- // Metadata regex.
(@WALT :wife /Lillian/ Mrs. Walter Elias)- x Metadata regex chaining or flags.
(@WALT :wife /Lillian/ Mrs. /Disney/ Walter Elias Disney)- [] Metadata bracketed transform.
(@WALT :wife [//drive/file.msl])- x Metadata bracketed transform multiple occurrences.
(@WALT :wife [//drive/file.msl] [//drive/file2.msl])- d Metadata datatype selector.
(@WALT :wife (d person))- x Metadata datatype multiple occurrences.
(@WALT :wife (d person) (d mother))- f Metadata format transform.
(@WALT :wife (f family-tree))- x Metadata format chaining.
(@WALT :wife (f family-tree) (f svg))
¶ Combining Bits to Show Coverage in Complex Expressions
All of the features covered by a complex MSL expression can be indicated by turning on the individual bits for each feature in the expression.
EXAMPLE
(@WALT Walt Disney :wife Lillian //key value meta value)This expression covers an atom, its value, a metadata key, and its value. Turning on only these bits in the test strip results in a binary value of 101000000000101000000000, or test #1048-8320, which represents all four features covered by the test. (Notice that four bits are turned on in the binary test strip value.)
¶ Using the Test Strip
Clicking a bit in the test strip generates the corresponding decimal test number on the row below the strip. If a test on the list has a matching number, the first test on the list with a matching number is recalled into the MSL Send box beneath the strip. The test strip control row consists of 6 parts:
- Indicates this is the test strip control row.
- Runs only the test on this row. The test strip history is kept separate from the test list history, though matching tests on the list will also illuminate to show pass/fail.
- Test number. Generated by clicking bits in the strip. Lights green if a test on the list matches this number, red if not. When editing a row, click to copy this test strip number to the editing row.
- Send. Displays the MSL expression to send from the matching test on the test list. This box cannot be edited.
- MSL Received. After the test strip test is run, displays the actual MSL received. This box cannot be edited.
- Admin Received. After the test strip test is run, displays the actual admin message received. This box cannot be edited.
¶ MSL Text
Nebula's MSL Text editor provides an alternate way to generate and edit tests in bulk.
REDUCED INFORMATION FORMAT
MSL Text contains less information than MSL tests in JSON form. If an empty test list is replaced by MSL text, the new list will be missing:
- Test numbers
- Notes
- Admin messages
- Breakpoints
- MSL Expected values
- Admin Expected values
To preserve all test information, save your test lists in JSON format using the list controls.
¶ Editing Text
When the MSL Text editor is open, an additional row appears beneath the list controls with 6 controls:
The MSL Text editor originally displays as a single line. Use the
resizecontrol at the bottom right of the MSL text box to make the box larger and display more lines.
- Line count. The number of lines in the MSL Text. This may vary from the lines in the test list if the list contains admin messages (which are removed in MSL text) or if the MSL text contains blank lines.
- Runs all the tests on the test list. Same as on the list controls row.
- Reset MSL text to the last saved version.
Shift Load MSL text from a file.- Save the MSL text to a file.
- Convert MSL Text to test list messages. Leaves other list aspects unchanged.
Shift Convert test list messages to MSL Text. Erases existing MSL text.- MSL Text. Use this box to compose or edit the MSL text.